1. Field of the Invention
The present invention relates to an image coding apparatus which converts an image into frequency areas and codes the image by adaptively performing quantization.
2. Description of the Related Art
More gradations and higher resolution are required to obtain higher quality digital images. The capacity of images is represented as the product of the number of pixels and the number of gradation bits, which represents an enormous amount of information. Therefore, an image size is compressed to reduce costs of storage or transmission of images.
A variety of image coding systems are proposed. A typical one is JPEG (Joint Photographic Experts Group) Baseline system described in Pages 18 to 23 of xe2x80x9cInternational Standards of Multimedia Codingxe2x80x9d edited by Yasuda, published by Maruzen. This system will be explained using FIG. 12.
In FIG. 12, the reference numerals 1, 2, 3, 4, 5, 6, 7, 8, 9, 10, 11, 12, 13, 14, and 15 designate an input image, a blocking circuit, a DCT (discrete cosine transform) circuit, a quantizer, a quantization table, a scan converting circuit, a significant coefficient detecting circuit, a grouping circuit, a run length counter, a two-dimensional Huffman coding circuit, a DC difference calculating circuit, a grouping circuit, a one-dimensional Huffman coding circuit, a multiplexing circuit, and an output code, respectively.
In FIG. 12, an input image 1 is split into blocks (hereinafter called pixel blocks) of 8xc3x978 pixels each by the blocking circuit 2. The pixel blocks are subjected to a DCT operation by the DCT circuit 3 and transform coefficients output as a result of the DCT operation are quantized by the quantizer 4 according to quantization step information stored in the quantization table 5. The quantized transform coefficients can be represented as an 8-by-8 matrix. Usually, the transform coefficients are arranged in the array so that the vertical coefficients correspond to higher-order DCT coefficients as they go downward and the horizontal coefficients correspond to higher-order DCT coefficients as they go toward the right. The leftmost and topmost of the 64 transform coefficients is called a DC component or DC coefficient because it corresponds to a DC frequency area of a DCT area. The other 63 are called AC components or AC coefficients because they correspond to AC frequency areas.
The DC difference calculating circuit 11 calculates a difference between the DC coefficient and a DC component of a preceding image block and sends it to the grouping circuit 12. The grouping circuit 12 calculates, from the DC difference value, a group number and additional bits shown in FIG. 15. The additional bits are used to identify a DC difference value within an identical group. The number of bits of the additional bits is shown in FIG. 15. The group number calculated in the grouping circuit 12 is converted into a Huffman code by the one-dimensional Huffman coding circuit 13. The additional bits are sent to the multiplexing circuit 14.
The AC coefficients quantized by the quantizer 4 are scan-converted in a zigzag scan order shown in FIG. 13 and are sent to the significant coefficient detecting circuit 7. The significant coefficient detecting circuit 7 judges whether the quantized AC coefficients are xe2x80x9c0xe2x80x9d or not, and when xe2x80x9c0xe2x80x9d, supplies a count-up signal to the run length counter 9 to increment the counter value by one. When the AC coefficients are nonzero significant coefficients, the significant coefficient detecting circuit 7 supplies a reset signal to the run length counter 9 to reset the counter value and sends the AC coefficients to the grouping circuit 8.
The run length counter 9 counts the length of a run of xe2x80x9c0xe2x80x9ds. The number NNNN of xe2x80x9c0xe2x80x9ds between two significant coefficients is sent to the two-dimensional Huffman coding circuit 10. The grouping circuit 8 splits the AC coefficients into group numbers SSSS and additional bits shown in FIG. 14, and sends the group numbers to the two-dimensional Huffman coding circuit 10 and the additional bits to the multiplexing circuit 14. The additional bits are used to identify a DC difference value within an identical group. The number of bits of the additional bits is shown in FIG. 14.
The two-dimensional Huffman coding circuit 10 converts a combination of a run length NNNN and a group number SSSS into a Huffman code and sends it to the multiplexing circuit 14.
The multiplexing circuit 14 multiplexes a DC coefficient Huffman code of one pixel block, an AC coefficient Huffman code, DC coefficient additional bits, and AC coefficient additional bits and outputs code data 15.
This terminates a description of the JPEG Baseline coding system.
Furthermore, by changing the contents of the quantization table 5 of the standard image coding system described above, the following effects can be obtained:
(1) The amount of code of an input image can be controlled to a desired level.
(2) Quantization resistant to image degradation can be performed in accordance with the nature of an input image.
The above described two points will be described in more detail.
The first point will be described. When a quantization step width is small, the amount of code increases because probably the number of significant coefficients will increase and the absolute values of the significant coefficients will become large. Conversely, when a quantization step value is large, the amount of code decreases because probably a run length will increase due to a small number of significant coefficients and the absolute values of the significant coefficients will become small. By providing a basic quantization table and multiplying a quantization step value within the basic quantization table by a constant called a scaling factor, the value of quantization step width can be increased or decreased. When the amount of code is to be increased to enhance image quality, the scaling factor should be reduced. When the amount of code is to be reduced, the scaling factor should be set to a large value.
Generally, a larger amount of code makes image quality better and a smaller amount of code makes image quality worse. However, the limit of storage capacity and transmission capacity may limit the amount of code. In this case, the best image quality can be obtained by controlling the amount of code to a limit.
Numerous methods of calculating a scaling factor with the objective of controlling the amount of code as described above are proposed in Japanese Published Unexamined Patent Application Nos. Hei 7-107296 and Hei 7-212757, and the like.
These methods are used to obtain a scaling factor of an entire input image. However, when an image size is large, there are cases where it is necessary to split the image into smaller partial images and control the amount of code for each of the partial images. In this case, if the split partial image is viewed as one image, these methods are also applicable to the control of code amount for each of the partial images.
Next, the second point will be described. Usually, halftone images produced from photographs input through an image scanner or the like tend to be power-intensive in low-frequency portions of blocks subjected to a DCT operation. Accordingly, in order to reduce an error power after quantization, the quantization table 5 is organized to have small step widths for low frequencies of DCT coefficients and large step widths for high frequencies.
However, in the edge portion (a point where a sudden change of density values or brightness values is found) of an image, power is distributed also in high-frequency portions of blocks subjected to a DCT operation. As is already known, quantizing an edge portion with a quantization table for halftone images causes image distortion called mosquito noise. To suppress the occurrence of the image distortion, a quantization table suitable for each of image parts should be selected for quantization. Alternatively, without changing a quantization table, the amount of code can be increased by reducing a scaling factor in the edge portion of an image, whereby the occurrence of image distortion can be suppressed.
Numerous methods of analyzing input images to select a quantization table so as to prevent the occurrence of image distortion are proposed in Japanese Published Unexamined Patent Application No. Hei 5-103208, and the like. Also, numerous methods of analyzing input images to select a scaling factor so as to prevent the occurrence of image distortion are proposed in Japanese Published Unexamined Patent Application Nos. Hei 7-288809, Hei 6-98307, Hei 5-145773, and Hei 8-265769, and the like.
As described above, when a scaling factor is changed or a quantization matrix is selected for each of parts, scaling factors or quantization matrix selection information must be coded for each of the parts. Without a method of coding scaling factors, it would be impossible to perform image coding operations which actually change scaling factors.
How to code scaling factors is not described in the conventional systems proposed in Japanese Published Unexamined Patent Application Nos. Hei 7-107296 and Hei 7-212757, and the like. Also, how to code a selected quantization table is not described in the conventional methods proposed in Japanese Published Unexamined Patent Application No. Hei 5-103208 and the like. Furthermore, the systems described in Japanese Published Unexamined Patent Application Nos. Hei 7-288809, Hei 6-98307, Hei 5-145773, and Hei 8-265769, and the like have the same method of coding the scaling factors as the MPEG (Moving Picture Coding Experts Group) system.
Hereinafter, a description will be made of conventional systems for coding scaling factors or quantization matrix selection information.
Conventional systems for coding scaling factors or quantization matrix selection information include:
(1) JPEG Extended system
(2) MPEG System
Hereinafter, the two conventional systems will be described.
The JPEG Extended system is described in ITU-T, T. 84, ISO/IEC 10918-3 (pages 32 and 33). This system makes it possible to change scaling factors for each pixel block. Coding is performed for each block in the following steps:
(S1): If a scaling factor is the same as that of a preceding block, the same coding as that of a normal JPEG system is performed.
(S2): If a scaling factor is different from that of a preceding block, in place of the code of DC difference, the code xe2x80x9cQS_CHANGExe2x80x9d to indicate change of the scaling factor is generated. xe2x80x9cQS_CHANGExe2x80x9d is followed by 5-bit information (SCALE_CODE) to indicate a scaling factor. Thereafter, the same coding as that of a normal JPEG system is performed. SCALE_CODE is 5-bit information as shown in FIG. 16, and either of the linear and non-linear tables is referenced to find a scaling factor (Q_SCALE). Letting a DCT coefficient be X, a basic quantization step S, and an quantization index Y, the quantization index is found by the following expression.
Y=Xxc3x9716/(Sxc3x97Q_SCALE)xe2x80x83xe2x80x83[Expression 1]
Hereinafter, referring to FIG. 17, a description is further made of an example of the operation of a conventional system. In FIG. 17, the same portions in FIG. 12 use the same reference numerals and will not be described. The reference numeral 51 designates an AC component coding circuit, which integrates the scan converting circuit 6, significant coefficient detecting circuit 7, grouping circuit 8, run length counter 9, and two-dimensional Huffman coding circuit 10. 52 denotes a pixel value of an input block, DCT coefficient, or the like, 53 a scaling factor calculating circuit, 54 a basic quantization table, 55 a scaling factor, and 56 a basic quantization step.
The scaling factor calculating circuit 53 calculates a scaling factor 55 and sends it to the quantizer 4 and the multiplexing circuit 14, as do the foregoing conventional systems, that is, those described in Japanese Published Unexamined Patent Application Nos. Hei 7-107296, Hei 7-212757, Hei 7-288809, Hei 6-98307, Hei 5-145773, and Hei 8-265769, and the like. Input information 52 can be pixel value information itself, DCT coefficient, or other information. The basic quantization table 54, which stores quantization step values, sends a quantization step value 56 to the quantizer 4. The quantizer 4 finds a quantization index by the following expression, letting a DCT coefficient be X, a basic quantization step S, a quantization index Y, and a scaling factor Q.
Y=Xxc3x9716/(Sxc3x97Q)xe2x80x83xe2x80x83[Expression 2]
Furthermore, as described above, the multiplexing circuit 14, if the scaling factor is different from that of a preceding block, in place of the code of DC difference, generates the code xe2x80x9cQS_CHANGExe2x80x9d to indicate change of the scaling factor. xe2x80x9cQS_CHANGExe2x80x9d is followed by 5-bit information (SCALE_CODE) to indicate a scaling factor. Thereafter, the same code as that of a normal JPEG system is appended.
Next, the MPEG system is described. The MPEG standard, which is a moving picture coding system, comprises numerous systems such as a method of coding a plurality of components, and a method of coding different frames. Herein, a description is made of a system of coding scaling factors. The system of coding scaling factors can also be applied to the coding of static pictures.
In the MPEG system, an input image (frame) is split into pixel blocks of 8xc3x978 pixels each. As shown in FIG. 18, a collection of a plurality of the split pixel blocks forms a macro block. Coding is performed for each macro block. A macro block type is appended as the header of a macro block. Macro block types fall into two categories, depending on whether QS is provided or not. The existence of QS indicates that a macro block type is followed by information (QS) for changing a scaling factor. The absence of QS indicates that information (QS) for changing a scaling factor is not provided. A macro block type is coded with a variable length. QS is the same as SCALE_CODE shown in FIG. 16 and is appended with five bits.
The MPEG system can also be described with reference to FIG. 17. The following description focuses on quantization by use of scaling factors and the coding of the scaling factors.
The scaling factor calculating circuit 53 calculates a scaling factor for each macro block. A calculating method can be performed as in the foregoing conventional systems, that is, those described in Japanese Published Unexamined Patent Application Nos. Hei 7-288809, Hei 6-98307, Hei 5-145773, and Hei 8-265769. The scaling factor 55 is sent to the quantizer 4 and the multiplexing circuit 14. Input information 52 can be pixel value information itself, DCT coefficient, or other information. The basic quantization table 54, which stores quantization step values, sends a quantization step value 56 to the quantizer 4. The quantizer 4 finds a quantization index by the following expression, letting a DCT coefficient be X, a basic quantization step S, a quantization index Y, and a scaling factor Q.
Y=Xxc3x9716/(Sxc3x97Q)xe2x80x83xe2x80x83[Expression 3]
Furthermore, the multiplexing circuit 14, when the scaling factor is different from that of a preceding macro block, generates, as a macro block type, a code indicating that QS is present, and appends 5-bit information (SCALE_CODE) indicating a scaling factor after that code. Thereafter, the macro block is coded. If the scaling factor is the same as that of a preceding block, a code indicating that QS is not present is generated, and thereafter the macro block is coded.
In the conventional systems, the higher the frequency with which scaling factors are changed, the more necessary it is to append scaling factor information to most pixel blocks or macro blocks. The amount of code will increase because of the large overhead.
Also, in the conventional systems, it has been impossible to code at the same time both information of scaling factor change and quantization matrix change information.
An object of the present invention is to achieve efficient coding by coding DC difference information and scaling factor information or quantization matrix change information, or scaling factor information and quantization matrix change information in combination.
The present invention makes coding more efficient by combined coding of DC difference values and scaling factors or quantization matrix selection information than by independent coding of them.
Hereinafter, the present invention will be described.
According to the present invention, in order to achieve the above object, an image coding apparatus is provided with an image splitting unit for splitting an input image; an orthogonal transformation unit for orthogonally transforming images split by the image splitting unit; a quantization step width control information output unit for outputting one or more types of quantization step width control information of the split images; a DC differential signal calculating unit for calculating a difference between a DC signal of signals orthogonally transformed by the orthogonal transformation unit and a DC signal of a split image split immediately before; and a DC differential signal and quantization step width control information coding unit for coding the DC differential signal and the one or more types of quantization step width control information, wherein the DC differential signal and quantization step width control information coding unit changes the code of the DC differential signal and at least one predetermined code of the respective codes of the one or more types of quantization step width control information in accordance with the value of the DC differential signal and at least one of the values of the one or more types of quantization step width control information which is not corresponding to the predetermined code(s).
The image splitting unit splits an input image into pixel blocks. The orthogonal transformation unit can split an input pixel block into DC components and AC components, such as discrete cosine transform (DCT), for example. The quantization step width control information denotes scaling factors, or quantization table selection information, and the like. The DC signal denotes DC coefficients in the case of DCT.
The DC differential signal and quantization step width control information coding unit may code DC differential signals and quantization step width control information singly as discrete codes or collectively as one code.
In this configuration, DC differential signals and quantization step width control information can be efficiently coded by expanding information sources or using the correlation between the information sources.
In this configuration, codes can be changed in the following states.
(1) Methods of coding scaling factors or quantization table selection information are switched by a DC difference value.
(2) Methods of coding DC difference values are switched by the value of a scaling factor or quantization table selection information.
(3) By the value of either of a scaling factor or quantization table selection information, methods of coding the other are switched.
(4) DC difference values, and scaling factors or quantization table selection information are collectively coded using the correlation among them.
The image coding apparatus, further provided with a basic quantization table storing unit, uses quantization step width control information as one variable numeric value and multiplies some or all values of a basic quantization table stored in the basic quantization table storing unit by the quantization step width control information, thereby calculating quantization step values used for quantization.
Briefly, quantization step control information can be used as a scaling factor. Multiplying some quantization step values of a basic quantization table stored in the basic quantization table storing unit by quantization step width control information means, for example, application of multiplication to only AC components but not to DC components.
The image coding apparatus, further provided with a basic quantization table storing unit for storing a plurality of basic quantization tables, can use, as quantization step width control information, information for selecting a quantization table used for quantization from a plurality of basic quantization tables stored in the basic quantization table storing unit. Briefly, quantization table selection information can be used as quantization step control information.
The DC differential signal and quantization step width control information coding unit can be configured to include a unit for grouping DC differential signals; a unit for degenerating N types of quantization step width control signals; and a multidimensional Huffman coding unit for converting a group number of DC differential signals and N types of quantization step width control degeneration signals degenerated by the quantization step width control signal degenerating unit into (N+1)-dimensional Huffman codes.
This configuration example intends to perform efficient coding taking advantage of the fact that the correlation is high between scaling factors and DC differences or the correlation between quantization table selection information and DC differences. For example, it is anticipated that a pixel block having a DC difference of 0 has almost the same nature as a preceding block. Accordingly, scaling factor difference information also will probably become 0. In two-dimensional Huffman coding, efficient coding can be performed by shortening codes which have a DC difference of 0 and scaling factor difference information of 0.
A quantization step width control degeneration signal is a signal generated by the following two types of processing, for example.
(S1): Calculate a difference between the scaling factor of a pixel block to be coded and the scaling factor of a preceding pixel block.
(S2): Group scaling factors like DC difference values and AC coefficients, split the scaling factors into a group number and additional bits, and convert the group number into a variable-length code.
The DC differential signal and quantization step width control information coding unit can be configured to include a DC differential signal threshold comparing unit; a DC differential signal coding unit; a quantization step width control signal degenerating unit; and a quantization step width control degeneration signal coding unit, wherein the quantization step width control degeneration signal coding unit performs variable-length coding while switching coding tables for coding quantization step width control degeneration signals, depending on a magnitude relation between the absolute value of a DC differential signal and a predetermined threshold value.
This configuration example takes advantage of the fact that the probability of the occurrence of the difference values of scaling factors and the difference values of quantization table selection information is different depending on DC difference values. For example, if a DC difference value is small, by using a coding table assuming that the difference values of scaling factors and the difference values of quantization table selection information are small, efficient coding is performed. If variable-length coding is Huffman coding, coding tables refer to Huffman tables.
The DC differential signal and quantization step width control information coding unit is configured to include a DC differential signal coding unit; a unit for detecting the states of N types of quantization step width control signals; a quantization step width control signal degenerating unit; and a quantization step width control degeneration signal coding unit, wherein the unit for detecting the states of N types of quantization step width control signals judges whether or not a quantization step width control signal is in a predetermined state. The DC differential signal coding unit performs variable-length coding while switching 2N types of coding tables, depending on in which of the two states N types of quantization step width control signals are. The quantization step width control degeneration signal coding unit, without coding a quantization step width control signal when it is in a predetermined state, codes the quantization step width control degeneration signal when the quantization step width control signal is not in a predetermined state.
In this configuration example, when a scaling factor is unchanged from that of a preceding block, no scaling factor information is appended and the efficiency of coding DC difference values can be increased.
A state of quantization step width control signal refers to, for example, a case where a scaling factor changes, or a case where it is unchanged. Another example is a case where quantization table selection information is 0 and a case where it is not 0.
A predetermined state refers to a state which will occur more frequently, of two states, such as a state in which a scaling factor is unchanged, or a state in which quantization table selection information is 0.
Specifically, two types of DC difference codes, one for a case where a scaling factor changes, and the other for a case where a scaling factor is unchanged, are provided to perform coding. In the case where scaling factors are unchanged, scaling factor information is not appended to codes. During decoding, it can be judged by DC difference codes whether or not a scaling factor is appended to codes.
The DC differential signal coding unit performs variable-length coding while switching 2N types of coding tables, depending on whether N types of quantization step width control signals are the same or different. All the 2N types of codes must exist in one type of code tree.
The quantization step width control signal degenerating unit can be configured to include a quantization step width control differential signal calculating unit. Specifically, differences between scaling factors are calculated and converted into variable-length codes.
Also, the quantization step width control signal degenerating unit can be configured to include a unit for grouping quantization step width control signals or quantization step width control differential signals. Specifically, scaling factors or scaling factor differences are grouped and the groups are converted into variable-length codes.
It is, of course, possible that differences between scale factors are calculated and grouped, and the groups are converted into variable-length codes.
The quantization step width control signal degenerating unit can also be configured to output quantization step width control signals without modifications. For example, the values themselves of scaling factors are coded.
The unit for grouping DC differential signals or the unit for grouping quantization step width control differential signals can also be configured to put a signal in a different group for a case where a difference value of the signal is positive and a case where it is negative. Correlation coefficients of DC differences and scaling factor differences are different for a case where DC difference values or scaling factor difference values are positive and a case where they are negative. By changing groups depending on whether they are positive or negative, the correlation can be used additionally. Conventionally, if absolute values are the same, the signals have been put in the same group.
The unit for grouping DC differential signals or the unit for grouping quantization step width control differential signals can also be configured to put small portions of the absolute value of a signal in an identical group regardless of whether the signal value is positive or negative, and put large portions of the absolute value of the signal in a different group for a case where the difference value of the signal is positive and a case where it is negative. For small portions of the absolute value of DC difference, it is anticipated that correlation values are unchanged regardless of whether the DC difference value is positive or negative. In this case, since it is useless to put the absolute value in different groups, small portions of the absolute value are put in an identical group.
According to the present invention, to achieve the above described object, an image coding apparatus is provided with an image splitting unit for splitting an input image; an orthogonal transformation unit for orthogonally transforming images split by the image splitting unit; a quantization step width control information output unit for outputting one or more types of quantization step width control information of the split images; a coding unit for coding signals orthogonally transformed by the orthogonal transformation unit; and a quantization step width control information coding unit, wherein the quantization step width control information coding unit includes a quantization step width control information change pattern creation unit for judging whether or not N types of quantization step width control signals have changed; a quantization step width control information change pattern coding unit for converting a quantization step width control information change pattern created by the quantization step width control information change pattern creation unit into variable-length codes; and a unit for coding N types of quantization step width control signals that, only when individual quantization step width control signals have changed, codes the quantization step width control signal.
In this configuration, since a plural items of quantization table control information are coded in such a way that only changed information is coded, efficient coding can be performed. Of course, efficient coding can be performed by expanding information sources or using the correlation between the information sources.